900612 ESP -- SURF Center : Extracting Image Point Features 2

This page last changed on May 17, 2012 by brian.

This page is deprecated and this technique is no longer used. We're retaining this page for archival purposes

Extracting Spots from ESP Images

The BIG Picture

The first step is to read the raw tiff image:

import java.io.File
import java.net.URL
import org.mbari.esp.ia.services.ToolBox
// READ ESP TIFF IMAGE
val imageIOService = ToolBox.imageIOService
val rawImage = imageIOService.read(new URL("file:/test07/hab11may0514h2000ml40s.tif")).getProcessor
// WRITE AS PNG 
imageIOService.write(rawImage, new File("hab11may0514h2000ml40s.png"))

There is a small function call that extracts the image spots. Behind the scenes it does a lot of things. I'll start with the big picture then we'll dive into the internals of how spots are extracted.

import org.mbari.esp.ia.imglib.SpotRegionsExtractorFn
val spotRegions = SpotRegionsExtractorFn(rawImage).filter(_.eccentricity()._1 < 3)
val imagePoints = spotRegions.map(_.centroid.toInt).toSeq

The above code snippet does the following:

Extracts bright regions of the image that are most likely to represent spots of interest.
- A region is just a collection of pixels that form a contiguous bright region
filter(_.eccentricity()._1 < 3) : This snippet excludes the least round spots. That is, it excludes regions that have odd shapes or ellipses as they are not likely to be spots of interest
spotRegions.map(_.centroid.toInt).toSeq Converts each region to a point by calculating the centroid of the region. The collection of regions is converted to a Seq which is roughly equivalent to a java.util.List

We can draw the extracted points onto the image using this bit of code:

import java.io.File
import java.awt.{Color, Graphics2D}
import java.awt.geom.{Ellipse2D}
import java.awt.image.BufferedImage
import org.mbari.esp.ia.imglib.ToRGBFn

val colorImage = ToRGBFn(rawImage).getBufferedImage
val g2 = colorImage.getGraphics.asInstanceOf[Graphics2D]
val extractedPoints = imagePoints.map(p =>
    new Ellipse2D.Double(p.x - 2, p.y - 2, 4, 4))
g2.setPaint(Color.GREEN)
extractedPoints.foreach(g2.draw(_))
imageIOService.write(colorImage, new File("hab11may0514h2000ml40s-with-extracted-spots.png"))
g2.dispose()

Under the Hood

So for simplicity, we would normally use a simple call to SpotRegionsExtractorFn to get the spots. Here's what's going on under the hood:

Apply a contrast function
- We make multiple passes over the rawImage applying different contrast treatments. Each treatment allows us to resolve different spots. We will walk through each treatment below.
Threshold the image
- This converts an image into a binary image of background (dark) areas and foreground (or bright) spots.
Flood fill the image
- The flood fill applies a unique id to each contiguous foreground region. This is how we id each spot.
Extract the spot regions

First Pass: Raw Image

import ij.process.ImageProcessor
import org.mbari.esp.ia.imglib.FloodFillWithMinimumFn
import org.mbari.esp.ia.imglib.MaximumEntropyThresholdFn
import org.mbari.esp.ia.imglib.RemoveNonFilterAreaFn
import org.mbari.esp.ia.imglib.SpotRegionsFn
import org.mbari.esp.ia.imglib.To16BitGrayFn
import org.mbari.esp.ia.imglib.To8BitGrayFn

// SETUP IMAGE PROCESSING FUNCTIONS
val threshold = To8BitGrayFn andThen MaximumEntropyThresholdFn
val floodFill = {
    val floodFillWMinFn = FloodFillWithMinimumFn(5, _: ImageProcessor)
    val removeFilterFn = RemoveNonFilterAreaFn(0.75, _: ImageProcessor)
    To16BitGrayFn andThen removeFilterFn andThen floodFillWMinFn
}

// SPOT EXTRACTION OF RAW IMAGE
val rawFloodFill = threshold andThen floodFill
val rawRegions = SpotRegionsFn(rawImage, rawFloodFill(rawImage)).filter(_.eccentricity()._1 < 3)
val rawPoints = rawRegions.map(_.centroid.toInt).toSeq

// CREATE IMAGE
val colorImage = ToRGBFn(rawImage).getBufferedImage
val g2 = colorImage.getGraphics.asInstanceOf[Graphics2D]
val extractedPoints = rawPoints.map(p =>
    new Ellipse2D.Double(p.x - 2, p.y - 2, 4, 4))
g2.setPaint(Color.GREEN)
extractedPoints.foreach(g2.draw(_))
imageIOService.write(colorImage, new File("hab11may0514h2000ml40s-with-extracted-raw-spots.png"))
g2.dispose()

Second Pass: Autocontrasted Image

import org.mbari.esp.ia.imglib.AutoContrastFn

// SPOT EXTRACTION OF AUTOCONTRASTED IMAGE
val autoContrastFn = new AutoContrastFn(0.95)
val acImage = autoContrastFn(rawImage)
val acRegions = SpotRegionsFn(rawImage, rawFloodFill(acImage)).filter(_.eccentricity()._1 < 3)
val acPoints = acRegions.map(_.centroid.toInt).toSeq

// CREATE IMAGE
val colorImage = ToRGBFn(acImage).getBufferedImage
val g2 = colorImage.getGraphics.asInstanceOf[Graphics2D]
val extractedPoints = acPoints.map(p =>
    new Ellipse2D.Double(p.x - 2, p.y - 2, 4, 4))
g2.setPaint(Color.GREEN)
extractedPoints.foreach(g2.draw(_))
imageIOService.write(colorImage, new File("hab11may0514h2000ml40s-with-extracted-ac-spots.png"))
g2.dispose()

Third Pass: Dynamic Contrast (Fuzzy-Sets)

import org.mbari.esp.ia.imglib.FuzzyContrastFn
val fuzzyContrastFn = To16BitGrayFn andThen FuzzyContrastFn
val fcImage = fuzzyContrastFn(rawImage)
val fcRegions = SpotRegionsFn(rawImage, rawFloodFill(fcImage)).filter(_.eccentricity()._1 < 3)
val fcPoints = fcRegions.map(_.centroid.toInt).toSeq

// CREATE IMAGE
val colorImage = ToRGBFn(fcImage).getBufferedImage
val g2 = colorImage.getGraphics.asInstanceOf[Graphics2D]
val extractedPoints = fcPoints.map(p =>
    new Ellipse2D.Double(p.x - 2, p.y - 2, 4, 4))
g2.setPaint(Color.GREEN)
extractedPoints.foreach(g2.draw(_))
imageIOService.write(colorImage, new File("hab11may0514h2000ml40s-with-extracted-fc-spots.png"))
g2.dispose()